Alex Crichton [Mon, 11 Apr 2016 19:55:38 +0000 (12:55 -0700)]
Update dependency on tar
Includes a fix for alexcrichton/tar-rs#61
bors [Fri, 8 Apr 2016 20:31:50 +0000 (13:31 -0700)]
Auto merge of #2554 - alexcrichton:robust-test, r=alexcrichton
Make a test a bit more robust to the environment
Don't filter out PATH as `rustc` may be in that directory (e.g. when using
rustup) and otherwise just have a longer subcommand.
Closes #2553
Alex Crichton [Fri, 8 Apr 2016 20:30:45 +0000 (13:30 -0700)]
Make a test a bit more robust to the environment
Don't filter out PATH as `rustc` may be in that directory (e.g. when using
rustup) and otherwise just have a longer subcommand.
Closes #2553
bors [Fri, 8 Apr 2016 18:53:24 +0000 (11:53 -0700)]
Auto merge of #2465 - TheNeikos:add-version_env, r=alexcrichton
Add a new CARGO_PKG_AUTHORS environment variable
This will allow crates to use the CARGO_PKG_AUTHORS env variable to get a comma
seperated list of the authors declared in the manifest.
Closes #2441
Marcel Müller [Thu, 10 Mar 2016 09:35:50 +0000 (10:35 +0100)]
Add a new CARGO_PKG_AUTHORS environment variable
This will allow crates to use the CARGO_PKG_AUTHORS env variable to get a colon
seperated list of the authors declared in the manifest.
Closes #2441
bors [Thu, 7 Apr 2016 21:17:56 +0000 (14:17 -0700)]
Auto merge of #2385 - alexcrichton:top-level-overrides, r=wycats
Implement top-level overrides
This commit is an implementation of top-level overrides to be encoded into the
manifest itself directly. This style of override is distinct from the existing
`paths` support in `.cargo/config` in two important ways:
* Top level overrides are intended to be checked in and shared amongst all
developers of a project.
* Top level overrides are reflected in `Cargo.lock`.
The second point is crucially important here as it will ensure that an override
on one machine behaves the same as an override on another machine. This solves
many long-standing problems with `paths`-based overrides which suffer from some
level of nondeterminism as they're not encoded.
From a syntactical point of view, an override looks like:
```toml
[replace]
"libc:0.2.0" = { git = 'https://github.com/my-username/libc';, branch = '0.2-fork' }
```
This declaration indicates that whenever resolution would otherwise encounter
the `libc` package version 0.2.0 from crates.io, it should instead replace it
with the custom git dependency on a specific branch.
The key "libc:0.2.0" here is actually a package id specification which will
allow selecting various components of a graph. For example the same named
package coming from two distinct locations can be selected against, as well as
multiple versions of one crate in a dependency graph. The replacement dependency
has the same syntax as the `[dependencies]` section of Cargo.toml.
One of the major uses of this syntax will be, for example, using a temporary
fork of a crate while the changes are pushed upstream to the original repo. This
will avoid the need to change the intermediate projects immediately, and over
time once fixes have landed upstream the `[replace]` section in a `Cargo.toml`
can be removed.
There are also two crucial restrictions on overrides.
* A crate with the name `foo` can only get overridden with packages also of
the name `foo`.
* A crate can only get overridden with a crate of the exact same version.
A consequence of these restrictions is that crates.io cannot be used to replace
anything from crates.io. There's only one version of something on crates.io, so
there's nothing else to replace it with (name/version are a unique key).
Closes #942
bors [Wed, 6 Apr 2016 16:56:36 +0000 (09:56 -0700)]
Auto merge of #2547 - alexcrichton:more-locking, r=brson
Add file locks in a few more locations
One was discovered by making `target_dir` return a `Filesystem`, the other was discovered by failing tests on the nightly OSX bots.
Alex Crichton [Tue, 5 Apr 2016 23:32:13 +0000 (16:32 -0700)]
Protect against concurrent access to Cargo.lock
CI tests seen to be spurious failing on OSX due to this, I believe it's because
one process is concurrently writing out Cargo.lock while the other is trying to
read it in, so one of them gets a corrupt version.
This would ideally be fixed from `pkg.root()` returning a `Filesystem`, but that
was unfortunately such an invasive change that it seemed untenable. Additionally
we generally don't write files into the source tree, so if this is the only
instance it's perhaps not so bad trying to be robust in to the future.
Alex Crichton [Sat, 2 Apr 2016 01:06:20 +0000 (18:06 -0700)]
Change `Config::target_dir` to return Filesystem
This is a shared directory among multiple Cargo processes so access to it needs
to be properly synchronized. This commit changes the API of `Config::target_dir`
and then propagates the changes outward as necessary.
One fallout of this change is now we allow release/debug builds to proceed in
parallel.
bors [Sun, 3 Apr 2016 01:17:51 +0000 (18:17 -0700)]
Auto merge of #2541 - alexcrichton:aarch64, r=alexcrichton
Download the aarch64 standard library on the bots
Alex Crichton [Sat, 2 Apr 2016 01:08:21 +0000 (18:08 -0700)]
Download the aarch64 standard library on the bots
bors [Sat, 2 Apr 2016 23:26:03 +0000 (16:26 -0700)]
Auto merge of #2534 - alexcrichton:lock-with-git-repos, r=brson
Replace existing sources before updating
Currently sources may acquire file locks to ensure that they're not tampered
with while they're in use. We may load two sources to the same location,
however, in the case of git repositories which need to be updated. Cargo will
first load a locked version of the source and then may load an unlocked version,
and these two loads currently deadlock.
This commit tweaks the logic when updating a source to only update it after the
previous source has been replaced.
Closes #2533
bors [Sat, 2 Apr 2016 01:13:23 +0000 (18:13 -0700)]
Auto merge of #2538 - alexcrichton:aarch64, r=alexcrichton
Prepare for aarch64 nightlies and CI
Alex Crichton [Sat, 2 Apr 2016 01:08:21 +0000 (18:08 -0700)]
Prepare for aarch64 nightlies and CI
bors [Fri, 1 Apr 2016 17:56:49 +0000 (10:56 -0700)]
Auto merge of #2505 - srinivasreddy:improve_msg, r=alexcrichton
Create `Cargo.lock` if it does not exist.
Srinivas Reddy Thatiparthy [Thu, 31 Mar 2016 17:32:14 +0000 (23:02 +0530)]
remote unnecessary expression
bors [Thu, 31 Mar 2016 16:37:48 +0000 (09:37 -0700)]
Auto merge of #2491 - JIghtuse:master, r=alexcrichton
Reject manifest with duplicate dependencies in different targets
Closes #2023
srinivasreddy [Sun, 20 Mar 2016 20:12:52 +0000 (01:42 +0530)]
Generate the lock file when there is no lock file
Alex Crichton [Wed, 30 Mar 2016 17:39:24 +0000 (10:39 -0700)]
Replace existing sources before updating
Currently sources may acquire file locks to ensure that they're not tampered
with while they're in use. We may load two sources to the same location,
however, in the case of git repositories which need to be updated. Cargo will
first load a locked version of the source and then may load an unlocked version,
and these two loads currently deadlock.
This commit tweaks the logic when updating a source to only update it after the
previous source has been replaced.
Closes #2533
bors [Wed, 30 Mar 2016 16:11:39 +0000 (09:11 -0700)]
Auto merge of #2532 - japaric:musl, r=alexcrichton
install-deps.py: add x86_64-unknown-linux-musl target
This lets me build a statically linked cargo on our linux-cross Docker image with the following
commands:
$ apt-get install musl-tools # for musl-gcc, which is needed to build openssl
$ ./.travis.install.deps.sh
$ ./configure --local-rust-root=$(pwd)/rustc --enable-nightly --target=x86_64-unknown-linux-musl
$ make
$ file target/x86_64-unknown-linux-musl/release/cargo
cargo: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked (..)
---
r? @alexcrichton -- If we want start building cargo for this target on the bots, we'll have to modify the linux-cross Dockerfile to install musl-tools.
cc @eddyb
bors [Wed, 30 Mar 2016 00:43:41 +0000 (17:43 -0700)]
Auto merge of #2531 - alexcrichton:fix-stdin, r=alexcrichton
Update Rust to pick up rust-lang/rust#32257
Closes #2530
Alex Crichton [Wed, 30 Mar 2016 00:41:18 +0000 (17:41 -0700)]
Update Rust to pick up rust-lang/rust#32257
Closes #2530
Jorge Aparicio [Wed, 30 Mar 2016 00:38:13 +0000 (19:38 -0500)]
install-deps.py: add x86_64-unknown-linux-musl target
This lets me build a statically linked cargo on our linux-cross Docker image with the following
commands:
$ apt-get install musl-tools # for musl-gcc, which is needed to build openssl
$ ./.travis.install.deps.sh
$ ./configure --local-rust-root=$(pwd)/rustc --enable-nightly --target=x86_64-unknown-linux-musl
$ make
$ file target/x86_64-unknown-linux-musl/release/cargo
cargo: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked (..)
bors [Tue, 29 Mar 2016 16:48:28 +0000 (09:48 -0700)]
Auto merge of #2523 - srinivasreddy:issue-2504, r=alexcrichton
expose cargo package name as CARGO_PKG_NAME
srinivasreddy [Sat, 26 Mar 2016 20:00:26 +0000 (01:30 +0530)]
implemented cargo package name as CARGO_PKG_NAME; And package description and home page are exposed as CARGO_PKG_DESCRIPTION and CARGO_PKG_HOMEPAGE respectively. And add a test case - for CARGO_PKG_NAME, CARGO_PKR_DESCRIPTION, CARGO_HOMEPAGE.
Alex Crichton [Tue, 9 Feb 2016 23:42:17 +0000 (15:42 -0800)]
Implement top-level overrides
This commit is an implementation of top-level overrides to be encoded into the
manifest itself directly. This style of override is distinct from the existing
`paths` support in `.cargo/config` in two important ways:
* Top level overrides are intended to be checked in and shared amongst all
developers of a project.
* Top level overrides are reflected in `Cargo.lock`.
The second point is crucially important here as it will ensure that an override
on one machine behaves the same as an override on another machine. This solves
many long-standing problems with `paths`-based overrides which suffer from some
level of nondeterminism as they're not encoded.
From a syntactical point of view, an override looks like:
```toml
[replace]
"libc:0.2.0" = { git = 'https://github.com/my-username/libc', branch = '0.2-fork' }
```
This declaration indicates that whenever resolution would otherwise encounter
the `libc` package version 0.2.0 from crates.io, it should instead replace it
with the custom git dependency on a specific branch.
The key "libc:0.2.0" here is actually a package id specification which will
allow selecting various components of a graph. For example the same named
package coming from two distinct locations can be selected against, as well as
multiple versions of one crate in a dependency graph. The replacement dependency
has the same syntax as the `[dependencies]` section of Cargo.toml.
One of the major uses of this syntax will be, for example, using a temporary
fork of a crate while the changes are pushed upstream to the original repo. This
will avoid the need to change the intermediate projects immediately, and over
time once fixes have landed upstream the `[replace]` section in a `Cargo.toml`
can be removed.
There are also two crucial restrictions on overrides.
* A crate with the name `foo` can only get overridden with packages also of
the name `foo`.
* A crate can only get overridden with a crate of the exact same version.
A consequence of these restrictions is that crates.io cannot be used to replace
anything from crates.io. There's only one version of something on crates.io, so
there's nothing else to replace it with (name/version are a unique key).
Closes #942
bors [Fri, 25 Mar 2016 16:26:34 +0000 (09:26 -0700)]
Auto merge of #2519 - alexcrichton:update-curl, r=alexcrichton
Update curl-sys
Picks up a fix to hopefully and correctly configure OpenSSL to be enabled in
cross-compiled situations where OpenSSL comes from a different location
(currently specified by the `OPENSSL_ROOT_DIR` environment variable that libssh2
also reads).
Alex Crichton [Fri, 25 Mar 2016 16:24:35 +0000 (09:24 -0700)]
Update curl-sys
Picks up a fix to hopefully and correctly configure OpenSSL to be enabled in
cross-compiled situations where OpenSSL comes from a different location
(currently specified by the `OPENSSL_ROOT_DIR` environment variable that libssh2
also reads).
bors [Thu, 24 Mar 2016 16:15:04 +0000 (09:15 -0700)]
Auto merge of #2513 - alexcrichton:xcompile, r=alexcrichton
Fix nightly dist builds
* When downloading rustc, also download a number of cross-std libraries so we
can cross compile with that compiler.
* Only build OpenSSL on some --enable-nightly builds, not all. For example
Windows and OSX don't want to link statically to OpenSSL.
Alex Crichton [Thu, 24 Mar 2016 01:07:19 +0000 (18:07 -0700)]
Fix nightly dist builds
* When downloading rustc, also download a number of cross-std libraries so we
can cross compile with that compiler.
* Only build OpenSSL on some --enable-nightly builds, not all. For example
Windows and OSX don't want to link statically to OpenSSL.
bors [Wed, 23 Mar 2016 16:36:36 +0000 (09:36 -0700)]
Auto merge of #2510 - alexcrichton:xcompile, r=brson
Prepare for ARM/FreeBSD/NetBSD nightlies
This commit beefs up Cargo's makefiles to support nightly builds of Cargo for
multiple platforms. This primarily involves vendoring the logic of how to build
OpenSSL for statically linking against Cargo into the Makefiles directly. We'll
have to update the version of OpenSSL as releases are made, but we essentially
already do that with the normal docker container.
The Linux nightlies will still run in the normal dist docker container (a really
old CentOS build) and builds for new platforms will happen in the standard
linux-cross container we use for other cross builds. The nightly versions of
these will produce Cargo tarballs for a whole bunch of platforms to get
uploaded.
This has been tested in the `alexcrichton/rust-slave-linux-cross:2016-03-17b`
docker container for the 3 ARM targets and FreeBSD target. NetBSD will come once
rust-lang/rust#32407 lands.
Alex Crichton [Tue, 22 Mar 2016 21:40:00 +0000 (14:40 -0700)]
Prepare for ARM/FreeBSD/NetBSD nightlies
This commit beefs up Cargo's makefiles to support nightly builds of Cargo for
multiple platforms. This primarily involves vendoring the logic of how to build
OpenSSL for statically linking against Cargo into the Makefiles directly. We'll
have to update the version of OpenSSL as releases are made, but we essentially
already do that with the normal docker container.
The Linux nightlies will still run in the normal dist docker container (a really
old CentOS build) and builds for new platforms will happen in the standard
linux-cross container we use for other cross builds. The nightly versions of
these will produce Cargo tarballs for a whole bunch of platforms to get
uploaded.
This has been tested in the `alexcrichton/rust-slave-linux-cross:2016-03-17b`
docker container for the 3 ARM targets and FreeBSD target. NetBSD will come once
rust-lang/rust#32407 lands.
bors [Tue, 22 Mar 2016 16:52:21 +0000 (09:52 -0700)]
Auto merge of #2440 - ThinTim:master, r=alexcrichton
Prevent the use of "test" as a crate name
Fixes #2432
?r @alexcrichton
Tim Erwich [Fri, 4 Mar 2016 19:32:22 +0000 (19:32 +0000)]
Prevent the use of "test" as a crate name. Fixes #2432.
Boris Egorov [Thu, 17 Mar 2016 20:04:33 +0000 (02:04 +0600)]
Reject manifest with duplicate dependencies in different targets
Closes #2023
bors [Sat, 19 Mar 2016 18:39:35 +0000 (11:39 -0700)]
Auto merge of #2502 - IvanUkhov:doc, r=alexcrichton
doc: make the pages’ titles consistent
The titles of some of the pages end with “Cargo Documentation” (_e.g._, [Frequently Asked Questions](http://doc.crates.io/faq.html)) whereas the titles of some other pages do not (_e.g._, [Environment Variables](http://doc.crates.io/environment-variables.html)), which is a bit inconsistent. Perhaps one should either add that ending to all the titles or eliminate it from all of them. This pull request does the latter, which can be changed if needed. I personally think that such long titles are reasonable for the `title` HTML tag but a bit too verbose when displayed on the page.
Ivan Ukhov [Sat, 19 Mar 2016 06:43:49 +0000 (07:43 +0100)]
doc: make the pages’ titles consistent
bors [Fri, 18 Mar 2016 22:02:02 +0000 (15:02 -0700)]
Auto merge of #2499 - IvanUkhov:typography, r=alexcrichton
doc/manifest: turn the list in Rules into an unordered one
Please see the comments in #2493.
Ivan Ukhov [Fri, 18 Mar 2016 18:28:45 +0000 (19:28 +0100)]
doc/manifest: turn the list in Rules into an unordered one
bors [Fri, 18 Mar 2016 18:07:20 +0000 (11:07 -0700)]
Auto merge of #2496 - alexcrichton:fix-concurrent, r=alexcrichton
Relax an assertion in the concurrent tests
The output may also have information about blocking, we just want the successful
part.
bors [Fri, 18 Mar 2016 17:49:04 +0000 (10:49 -0700)]
Auto merge of #2482 - ryanq:issue2266, r=alexcrichton
Suggest the best matching target for cargo run
Targets passed to cargo compile are validated against the package. If
the target exists, it is compiled. If not, cargo will bail and offer a
suggested target name if there is a close match.
The tests create and build/run binaries and examples using filenames
that are close (or not so close) to the target names to verify that
close matching names are suggested to the user.
Sample output attached.
<img width="838" alt="screen shot 2016-03-14 at 12 32 59 pm" src="https://cloud.githubusercontent.com/assets/124247/
13754913/
28490e42-e9ef-11e5-9617-
8d3c01106527.png">
Closes #2266
Alex Crichton [Fri, 18 Mar 2016 16:49:24 +0000 (09:49 -0700)]
Relax an assertion in the concurrent tests
The output may also have information about blocking, we just want the successful
part.
bors [Fri, 18 Mar 2016 16:47:42 +0000 (09:47 -0700)]
Auto merge of #2493 - IvanUkhov:typography, r=alexcrichton
doc/manifest: polish typography
The pull request makes a number of stylistic adjustments to The Manifest Format page. There are two commits. The first is a preliminary reflowing of paragraphs, and the second bears the actual changes. If any of the changes seems dubious, please let me know; I’ll try to motivate (and revert if needed).
Ryan Quattlebaum [Fri, 18 Mar 2016 16:39:14 +0000 (12:39 -0400)]
Fixing line length error in make test
Ryan Quattlebaum [Fri, 18 Mar 2016 16:20:36 +0000 (12:20 -0400)]
Bring in changes from master
Ryan Quattlebaum [Fri, 18 Mar 2016 16:00:22 +0000 (12:00 -0400)]
Move suggestion logic into generate_targets()
Ivan Ukhov [Fri, 18 Mar 2016 10:23:48 +0000 (11:23 +0100)]
doc/manifest: polish typography
Ivan Ukhov [Fri, 18 Mar 2016 10:06:01 +0000 (11:06 +0100)]
doc/manifest: respect 80 characters
bors [Thu, 17 Mar 2016 07:01:44 +0000 (00:01 -0700)]
Auto merge of #2467 - tshepang:rustfmt, r=alexcrichton
run rustfmt on core/source.rs
bors [Thu, 17 Mar 2016 06:43:11 +0000 (23:43 -0700)]
Auto merge of #2480 - alexcrichton:ui, r=alexcrichton
Tweak UI for warnings and errors
Right now Cargo prints out errors justified like all other status messages, but
this can look odd without coloration:
```
error some error message
```
Instead, this commit changes both warnings and errors to use the same style of:
```
error: some error message
warning: some warning message
```
Additionally, warnings now only print out "warning" in yellow instead of the
entire message (like errors do)
Alex Crichton [Sun, 13 Mar 2016 22:24:55 +0000 (15:24 -0700)]
Tweak UI for warnings and errors
Right now Cargo prints out errors justified like all other status messages, but
this can look odd without coloration:
```
error some error message
```
Instead, this commit changes both warnings and errors to use the same style of:
```
error: some error message
warning: some warning message
```
Additionally, warnings now only print out "warning" in yellow instead of the
entire message (like errors do)
Tshepang Lekhonkhobe [Thu, 10 Mar 2016 21:26:45 +0000 (23:26 +0200)]
run rustfmt on core/source.rs
Includes manual adjustments
bors [Thu, 17 Mar 2016 05:14:36 +0000 (22:14 -0700)]
Auto merge of #2486 - alexcrichton:flock, r=brson
Fix running Cargo concurrently
Cargo has historically had no protections against running it concurrently. This
is pretty unfortunate, however, as it essentially just means that you can only
run one instance of Cargo at a time **globally on a system**.
An "easy solution" to this would be the use of file locks, except they need to
be applied judiciously. It'd be a pretty bad experience to just lock the entire
system globally for Cargo (although it would work), but otherwise Cargo must be
principled how it accesses the filesystem to ensure that locks are properly
held. This commit intends to solve all of these problems.
A new utility module is added to cargo, `util::flock`, which contains two types:
* `FileLock` - a locked version of a `File`. This RAII guard will unlock the
lock on `Drop` and I/O can be performed through this object. The actual
underlying `Path` can be read from this object as well.
* `Filesystem` - an unlocked representation of a `Path`. There is no "safe"
method to access the underlying path without locking a file on the filesystem
first.
Built on the [fs2] library, these locks use the `flock` system call on Unix and
`LockFileEx` on Windows. Although file locking on Unix is [documented as not so
great][unix-bad], but largely only because of NFS, these are just advisory, and
there's no byte-range locking. These issues don't necessarily plague Cargo,
however, so we should try to leverage them. On both Windows and Unix the file
locks are released when the underlying OS handle is closed, which means that
if the process dies the locks are released.
Cargo has a number of global resources which it now needs to lock, and the
strategy is done in a fairly straightforward way:
* Each registry's index contains one lock (a dotfile in the index). Updating the
index requires a read/write lock while reading the index requires a shared
lock. This should allow each process to ensure a registry update happens while
not blocking out others for an unnecessarily long time. Additionally any
number of processes can read the index.
* When downloading crates, each downloaded crate is individually locked. A lock
for the downloaded crate implies a lock on the output directory as well.
Because downloaded crates are immutable, once the downloaded directory exists
the lock is no longer needed as it won't be modified, so it can be released.
This granularity of locking allows multiple Cargo instances to download
dependencies in parallel.
* Git repositories have separate locks for the database and for the project
checkout. The datbase and checkout are locked for read/write access when an
update is performed, and the lock of the checkout is held for the entire
lifetime of the git source. This is done to ensure that any other Cargo
processes must wait while we use the git repository. Unfortunately there's
just not that much parallelism here.
* Binaries managed by `cargo install` are locked by the local metadata file that
Cargo manages. This is relatively straightforward.
* The actual artifact output directory is just globally locked for the entire
build. It's hypothesized that running Cargo concurrently in *one directory* is
less of a feature needed rather than running multiple instances of Cargo
globally (for now at least). It would be possible to have finer grained
locking here, but that can likely be deferred to a future PR.
So with all of this infrastructure in place, Cargo is now ready to grab some
locks and ensure that you can call it concurrently anywhere at any time and
everything always works out as one might expect.
One interesting question, however, is what does Cargo do on contention? On one
hand Cargo could immediately abort, but this would lead to a pretty poor UI as
any Cargo process on the system could kick out any other. Instead this PR takes
a more nuanced approach.
* First, all locks are attempted to be acquired (a "try lock"). If this
succeeds, we're done.
* Next, Cargo prints a message to the console that it's going to block waiting
for a lock. This is done because it's indeterminate how long Cargo will wait
for the lock to become available, and most long-lasting operations in Cargo
have a message printed for them.
* Finally, a blocking acquisition of the lock is issued and we wait for it to
become available.
So all in all this should help Cargo fix any future concurrency bugs with file
locking in a principled fashion while also allowing concurrent Cargo processes
to proceed reasonably across the system.
[fs2]: https://github.com/danburkert/fs2-rs
[unix-bad]: http://0pointer.de/blog/projects/locking.html
Closes #354
Alex Crichton [Sat, 12 Mar 2016 17:58:53 +0000 (09:58 -0800)]
Fix running Cargo concurrently
Cargo has historically had no protections against running it concurrently. This
is pretty unfortunate, however, as it essentially just means that you can only
run one instance of Cargo at a time **globally on a system**.
An "easy solution" to this would be the use of file locks, except they need to
be applied judiciously. It'd be a pretty bad experience to just lock the entire
system globally for Cargo (although it would work), but otherwise Cargo must be
principled how it accesses the filesystem to ensure that locks are properly
held. This commit intends to solve all of these problems.
A new utility module is added to cargo, `util::flock`, which contains two types:
* `FileLock` - a locked version of a `File`. This RAII guard will unlock the
lock on `Drop` and I/O can be performed through this object. The actual
underlying `Path` can be read from this object as well.
* `Filesystem` - an unlocked representation of a `Path`. There is no "safe"
method to access the underlying path without locking a file on the filesystem
first.
Built on the [fs2] library, these locks use the `flock` system call on Unix and
`LockFileEx` on Windows. Although file locking on Unix is [documented as not so
great][unix-bad], but largely only because of NFS, these are just advisory, and
there's no byte-range locking. These issues don't necessarily plague Cargo,
however, so we should try to leverage them. On both Windows and Unix the file
locks are released when the underlying OS handle is closed, which means that
if the process dies the locks are released.
Cargo has a number of global resources which it now needs to lock, and the
strategy is done in a fairly straightforward way:
* Each registry's index contains one lock (a dotfile in the index). Updating the
index requires a read/write lock while reading the index requires a shared
lock. This should allow each process to ensure a registry update happens while
not blocking out others for an unnecessarily long time. Additionally any
number of processes can read the index.
* When downloading crates, each downloaded crate is individually locked. A lock
for the downloaded crate implies a lock on the output directory as well.
Because downloaded crates are immutable, once the downloaded directory exists
the lock is no longer needed as it won't be modified, so it can be released.
This granularity of locking allows multiple Cargo instances to download
dependencies in parallel.
* Git repositories have separate locks for the database and for the project
checkout. The datbase and checkout are locked for read/write access when an
update is performed, and the lock of the checkout is held for the entire
lifetime of the git source. This is done to ensure that any other Cargo
processes must wait while we use the git repository. Unfortunately there's
just not that much parallelism here.
* Binaries managed by `cargo install` are locked by the local metadata file that
Cargo manages. This is relatively straightforward.
* The actual artifact output directory is just globally locked for the entire
build. It's hypothesized that running Cargo concurrently in *one directory* is
less of a feature needed rather than running multiple instances of Cargo
globally (for now at least). It would be possible to have finer grained
locking here, but that can likely be deferred to a future PR.
So with all of this infrastructure in place, Cargo is now ready to grab some
locks and ensure that you can call it concurrently anywhere at any time and
everything always works out as one might expect.
One interesting question, however, is what does Cargo do on contention? On one
hand Cargo could immediately abort, but this would lead to a pretty poor UI as
any Cargo process on the system could kick out any other. Instead this PR takes
a more nuanced approach.
* First, all locks are attempted to be acquired (a "try lock"). If this
succeeds, we're done.
* Next, Cargo prints a message to the console that it's going to block waiting
for a lock. This is done because it's indeterminate how long Cargo will wait
for the lock to become available, and most long-lasting operations in Cargo
have a message printed for them.
* Finally, a blocking acquisition of the lock is issued and we wait for it to
become available.
So all in all this should help Cargo fix any future concurrency bugs with file
locking in a principled fashion while also allowing concurrent Cargo processes
to proceed reasonably across the system.
[fs2]: https://github.com/danburkert/fs2-rs
[unix-bad]: http://0pointer.de/blog/projects/locking.html
Closes #354
bors [Thu, 17 Mar 2016 01:12:32 +0000 (18:12 -0700)]
Auto merge of #2490 - alexcrichton:rustflags-docs, r=alexcrichton
Add docs for RUSTFLAGS and build.rustflags
bors [Thu, 17 Mar 2016 00:50:37 +0000 (17:50 -0700)]
Auto merge of #2484 - alexcrichton:fix-bad-backtrack, r=brson
Fix caching features across backtracking
In the local loop during resolution all variables need to be reset whenever we
backtrack up a frame, but currently the `method` and `features` set are
accidentally not reset whenever we backtrack. Calculate the `method` later and
cache `features` in each frame so we can properly backtrack.
Closes #2472
Alex Crichton [Thu, 17 Mar 2016 00:45:18 +0000 (17:45 -0700)]
Add docs for RUSTFLAGS and build.rustflags
bors [Thu, 17 Mar 2016 00:18:53 +0000 (17:18 -0700)]
Auto merge of #2241 - brson:rustflags, r=alexcrichton
Apply RUSTFLAGS arguments to rustc builds
Cargo will use RUSTFLAGS for building everything that is not a build script
or plugin. It does not apply to these targets because they may be for
a different platform that 'normal' builds.
Closes #2112
Alex Crichton [Mon, 14 Mar 2016 22:45:05 +0000 (15:45 -0700)]
Fix caching features across backtracking
In the local loop during resolution all variables need to be reset whenever we
backtrack up a frame, but currently the `method` and `features` set are
accidentally not reset whenever we backtrack. Calculate the `method` later and
cache `features` in each frame so we can properly backtrack.
Closes #2472
Brian Anderson [Thu, 3 Mar 2016 22:52:45 +0000 (22:52 +0000)]
Add rustflags support to config files
`build.rustflags` is treated exactly like `RUSTFLAGS`.
It is a list, so argument lists with spaces work.
`RUSTFLAGS` takes precedent, then `build.rustflags`.
Brian Anderson [Wed, 17 Feb 2016 01:14:49 +0000 (01:14 +0000)]
Recompile when RUSTFLAGS changes
Brian Anderson [Wed, 17 Feb 2016 00:48:03 +0000 (00:48 +0000)]
Apply RUSTFLAGS env var to rustc builds
This passes RUSTFLAGS to rustc builds for the target architecture.
We don't want to pass the RUSTFLAGS args to multiple architectures because
they may contain architecture-specific flags. Ideally, the scheme
we would use would treat plugins and build scripts - which may not
be for the target architecture - consistently. Unfortunately it's
quite difficult in the current Cargo architecture to seperately
identify build scripts, plugins and their dependencies from
code used by the target.
So the scheme here is very simple:
1) If --target is not specified, RUSTFLAGS applies to all builds.
2) If --target is specified, RUSTFLAGS only applies to builds
with the Kind::Target target kind, which indicates build units
derived from the requested --target.
Closes #2112
bors [Tue, 15 Mar 2016 16:57:46 +0000 (09:57 -0700)]
Auto merge of #2485 - japaric:no-cross-doctests, r=alexcrichton
don't build/run doctests when target != host
fixes rust-lang/rust#31907
r? @alexcrichton
Jorge Aparicio [Tue, 15 Mar 2016 15:35:52 +0000 (10:35 -0500)]
remove unused format argument
Jorge Aparicio [Tue, 15 Mar 2016 12:55:01 +0000 (07:55 -0500)]
update output of cross_tests
Jorge Aparicio [Tue, 15 Mar 2016 00:00:39 +0000 (19:00 -0500)]
don't build/run doctests when target != host
fixes rust-lang/rust#31907
bors [Mon, 14 Mar 2016 20:13:04 +0000 (13:13 -0700)]
Auto merge of #2483 - srinivasreddy:docs, r=alexcrichton
corrected statement regarding constraint graph solving problem type
srinivasreddy [Mon, 14 Mar 2016 19:26:26 +0000 (00:56 +0530)]
corrected statement regarding constraintgraph solving problem type
Ryan Quattlebaum [Mon, 14 Mar 2016 19:00:58 +0000 (15:00 -0400)]
Fix bug in target lookup
The target lookup used to only look for bin targets, but now looks for
targets with the correct type.
Ryan Quattlebaum [Mon, 14 Mar 2016 17:22:17 +0000 (13:22 -0400)]
Suggest the best matching target for cargo run
Targets passed to cargo compile are validated against the package. If
the target exists, it is compiled. If not, cargo will bail and offer a
suggested target name if there is a close match.
The tests create and build/run binaries and examples using filenames
that are close (or not so close) to the target names to verify that
close matching names are suggested to the user.
bors [Mon, 14 Mar 2016 17:26:19 +0000 (10:26 -0700)]
Auto merge of #2481 - jespino:add-favicon-to-doc, r=alexcrichton
Add favicon to doc.crates.io
This fix the issue rust-lang/crates.io#245
bors [Mon, 14 Mar 2016 17:08:12 +0000 (10:08 -0700)]
Auto merge of #2468 - TheNeikos:add-warning_if_no_browser, r=alexcrichton
Add warning if no browser
Closes #2371
I am unsure if `println!` is the correct way to print warnings at this stage, since it is not a hard error thus returning `Err` seems a bit too strong.
Jesús Espino [Mon, 14 Mar 2016 06:41:24 +0000 (07:41 +0100)]
Add favicon to doc.crates.io
Marcel Müller [Sun, 13 Mar 2016 16:18:09 +0000 (17:18 +0100)]
Don't include env in windows/macos
bors [Sat, 12 Mar 2016 20:34:58 +0000 (12:34 -0800)]
Auto merge of #2474 - sbeckeriv:add-some-flair, r=alexcrichton
Add build flair
Dearest Reviewer
I have add the travis-ci build badge to the README. I pushed the image down towards the bottom. I was thinking that most people reading the readme do not care about the build status. I have seen the badge done in different locations and with different titles. I am easy on where it goes. I added it because I found that I was looking for the status when my branch was failing. I have since learned the travis interface.
<3 Becker
<img width="415" alt="screen shot 2016-03-12 at 10 51 43 am" src="https://cloud.githubusercontent.com/assets/12170/
13724615/
7fe41e36-e840-11e5-8815-
3ac1f13dc2fd.png">
Stephen Becker IV [Sat, 12 Mar 2016 18:40:03 +0000 (10:40 -0800)]
Add flair to readme
Add the travis-ci and appveyor build badges to the readme
bors [Sat, 12 Mar 2016 20:08:12 +0000 (12:08 -0800)]
Auto merge of #2421 - sbeckeriv:decolor-messages-426, r=alexcrichton
Dull the errors
This resolves #426
Dearest Reviewer,
I have updated the error messages to use say_status at the shell level. I have also changed say_status to print the message in bold. I do think it looks nice but it does have the side effect of making some seemingly unrelated text bold. I do think it looks better bold but it is also very easy to revert. I have included examples of both.
Thank you,
Becker
Bold: Note the usage is bold.
<img width="1072" alt="screen shot 2016-02-27 at 10 49 05 am" src="https://cloud.githubusercontent.com/assets/12170/
13374778/
0efd54ec-dd43-11e5-9f02-
f0224608132a.png">
No bold:
<img width="885" alt="screen shot 2016-02-27 at 10 46 35 am" src="https://cloud.githubusercontent.com/assets/12170/
13374775/
fa3a6612-dd42-11e5-9c09-
8f23506f5f0c.png">
Both rendered on a mac with iterm 2.9.0
Stephen Becker IV [Sat, 27 Feb 2016 18:53:30 +0000 (10:53 -0800)]
Remove color from the errors
I updated the error states to use say_status.
Add text to the empty error
The empty error looked odd with the say_status change.
Update all stderr messages
Switch them to format statements and create a helper for the error
status.
bors [Sat, 12 Mar 2016 18:10:37 +0000 (10:10 -0800)]
Auto merge of #2473 - alexcrichton:update-curl, r=alexcrichton
Update curl dependency
Fixes a segfault on nightly
Alex Crichton [Sat, 12 Mar 2016 17:59:16 +0000 (09:59 -0800)]
Update curl dependency
Fixes a segfault on nightly
Marcel Müller [Sat, 12 Mar 2016 10:15:24 +0000 (11:15 +0100)]
Ignore $BROWSER if not set
Marcel Müller [Sat, 12 Mar 2016 09:53:41 +0000 (10:53 +0100)]
Show a list of tried browsers in the warning
bors [Sat, 12 Mar 2016 00:45:55 +0000 (16:45 -0800)]
Auto merge of #2471 - alexcrichton:install-git-lockfile, r=brson
Fix installing git repos with lockfiles
Just an erroneous assertion that needs to be ignored.
Closes #2466
bors [Sat, 12 Mar 2016 00:35:41 +0000 (16:35 -0800)]
Auto merge of #2454 - alexcrichton:less-recurse, r=brson
Globally optimize traversal in resolve
Currently when we're attempting to resolve a dependency graph we locally
optimize the order in which we visit candidates for a resolution (most
constrained first). Once a version is activated, however, it will add a whole
mess of new dependencies that need to be activated to the global list, currently
appended at the end.
This unfortunately can lead to pathological behavior. By always popping from the
back and appending to the back of pending dependencies, super constrained
dependencies in the front end up not getting visited for quite awhile. This in
turn can cause Cargo to appear to hang for quite awhile as it's so aggressively
backtracking.
This commit switches the list of dependencies-to-activate from a `Vec` to a
`BinaryHeap`. The heap is sorted by the number of candidates for each
dependency, with the least candidates first. This ends up massively cutting down
on resolution times in practice whenever `=` dependencies are encountered
because they are resolved almost immediately instead of way near the end if
they're at the wrong place in the graph.
This alteration in traversal order ended up messing up the existing cycle
detection, so that was just removed entirely from resolution and moved to its
own dedicated pass.
Closes #2090
bors [Fri, 11 Mar 2016 23:48:59 +0000 (15:48 -0800)]
Auto merge of #2470 - alexcrichton:better-path-error-message, r=brson
Improve the error message for missing path overrides
Closes #2457
Alex Crichton [Fri, 11 Mar 2016 20:18:17 +0000 (12:18 -0800)]
Fix installing git repos with lockfiles
Just an erroneous assertion that needs to be ignored.
Closes #2466
Alex Crichton [Fri, 11 Mar 2016 20:05:58 +0000 (12:05 -0800)]
Improve the error message for missing path overrides
Closes #2457
Marcel Müller [Fri, 11 Mar 2016 12:28:15 +0000 (13:28 +0100)]
Warn if could not open docs in browser
Marcel Müller [Fri, 11 Mar 2016 12:22:36 +0000 (13:22 +0100)]
Add $BROWSER as possible client
bors [Thu, 10 Mar 2016 01:18:27 +0000 (17:18 -0800)]
Auto merge of #2448 - alexcrichton:docs-dirty, r=brson
Fix rerunning rustdoc when output deleted
If `crate/index.html` is missing, we need to rerun rustdoc!
Closes #2379
bors [Wed, 9 Mar 2016 21:05:23 +0000 (13:05 -0800)]
Auto merge of #2420 - alexcrichton:different-metadata, r=brson
Ensure metadata for libs/bins are distinct
It may be the case in the future that the compiler will require that the "salt"
(the `-C metadata` flag) for all crates with the same name are distinct. Right
now a Cargo project with a library and a binary, however, will have the same
salt with the same crate name.
This commit mixes in some extra data to the library's salt to ensure that its
symbols don't clash with the binary's.
bors [Wed, 9 Mar 2016 20:01:05 +0000 (12:01 -0800)]
Auto merge of #2460 - jespino:add-limit-to-autocomplete, r=alexcrichton
Add the --limit parameter to search autocomplete
bors [Wed, 9 Mar 2016 19:39:28 +0000 (11:39 -0800)]
Auto merge of #2461 - jespino:adding-manpages-info, r=alexcrichton
Add search command to the manpage
Jesús Espino [Wed, 9 Mar 2016 18:53:11 +0000 (19:53 +0100)]
Add search command to the manpage
Jesús Espino [Wed, 9 Mar 2016 18:14:52 +0000 (19:14 +0100)]
Add the --limit parameter to search autocomplete
bors [Wed, 9 Mar 2016 17:33:06 +0000 (09:33 -0800)]
Auto merge of #2456 - jespino:remove-deprecated-code, r=alexcrichton
Remove deprecated usage of SliceConcatExt::connect
I think this can be changed to .join (the deprecation was in the 1.3 version)
bors [Wed, 9 Mar 2016 17:20:11 +0000 (09:20 -0800)]
Auto merge of #2455 - jespino:remove-completed-todo, r=alexcrichton
Removing finished TODO
I think this TODO is finished (The only public field in all the file is the Layout::path) If this is part of the TODO, i can set it as private and create a getter).
Jesús Espino [Wed, 9 Mar 2016 08:12:57 +0000 (09:12 +0100)]
Remove deprecated usage of SliceConcatExt::connect
Jesús Espino [Wed, 9 Mar 2016 08:18:23 +0000 (09:18 +0100)]
Removing finished TODO
Alex Crichton [Wed, 9 Mar 2016 00:37:00 +0000 (16:37 -0800)]
Globally optimize traversal in resolve
Currently when we're attempting to resolve a dependency graph we locally
optimize the order in which we visit candidates for a resolution (most
constrained first). Once a version is activated, however, it will add a whole
mess of new dependencies that need to be activated to the global list, currently
appended at the end.
This unfortunately can lead to pathological behavior. By always popping from the
back and appending to the back of pending dependencies, super constrained
dependencies in the front end up not getting visited for quite awhile. This in
turn can cause Cargo to appear to hang for quite awhile as it's so aggressively
backtracking.
This commit switches the list of dependencies-to-activate from a `Vec` to a
`BinaryHeap`. The heap is sorted by the number of candidates for each
dependency, with the least candidates first. This ends up massively cutting down
on resolution times in practice whenever `=` dependencies are encountered
because they are resolved almost immediately instead of way near the end if
they're at the wrong place in the graph.
This alteration in traversal order ended up messing up the existing cycle
detection, so that was just removed entirely from resolution and moved to its
own dedicated pass.
Closes #2090